System and method for displaying an image stream

ABSTRACT

A system and method may display an image stream, where an original image stream may be divided into two or more subset images streams, each subset image stream being displayed simultaneously or substantially simultaneously. The images may be displayed fused. The images may be collected from an ingestible capsule traversing the GI tract.

CROSS-REFERENCE TO RELATED APPLICATION

This application is continuation of patent application Ser. No.10/610,915, filed Jul. 2, 2003, entitled “System and Method forDisplaying an Image Stream”, which in turn is a continuation-in-part ofpatent application Ser. No. 10/364,508, filed Feb. 12, 2003, entitled“System and Method for Displaying an Image Stream”, which in turn claimsthe benefit of provisional application No. 60/355,796 filed Feb. 12,2002 entitled “System and Method for Viewing a Moving Image”, each ofwhich are incorporated by reference herein in their entirety.

FIELD OF THE INVENTION

The present invention relates to a method and system for displayingand/or reviewing image streams. More specifically, the present inventionrelates to a method and system for effective displaying of an imagestream.

BACKGROUND OF THE INVENTION

An image stream may be assembled from a series of still images anddisplayed to a user. The images may be created or collected from varioussources. For example, U.S. Pat. No. 5,604,531 to Iddan et al., assignedto the common assignee of the present application and incorporatedherein by reference, teaches an in-vivo imager system which in oneembodiment includes a swallowable capsule. The imager system capturesimages of a lumen such as the gastrointestinal (GI) tract and transmitsthem to an external recording device while the capsule passes throughthe lumen. Large numbers of images may be collected for viewing and, forexample, combined in sequence. An image stream of, for example, 40minutes in length, containing for example about 4,800 frames, may bepresented to the user for review. Other numbers of frames or lengths maybe used. In one embodiment, a streaming rate is preset, but the user canincrease or decrease the streaming rate at anytime during the reviewprocess, and/or define a different streaming rate. In general, a usermay try to set the streaming rate to the highest rate where the user canquickly and effectively review the image stream without missingimportant information that may be present in any of the images includedin the stream. The rate at which a user can effectively review a imagestream is limited by a physiological averaging effect that is known toexist at around 15 frames per second (although this number varies fordifferent users and image streams) above which certain details inindividual images displayed in the stream may be physiologicallyfiltered out.

Therefore, a need exists for a system and method that enables a user toincrease the rate at which the user can efficiently and effectivelyreview an image stream.

SUMMARY OF THE INVENTION

In one embodiment, a system and method are provided for displaying animage stream, where an original image stream is divided into two or moreimages streams, each image stream being displayed simultaneously orsubstantially simultaneously. According to another embodiment of theinvention, two or more subset image streams are displayed simultaneouslyor substantially simultaneously as a single entity stream.

When used herein, “substantially simultaneously” includes simultaneouslyand almost simultaneously. A system and method according to oneembodiment of the invention enables a user to see images in the streamfor a longer period of time without increasing the overall viewing timeof the entire image stream. Alternatively, the system and methoddescribed according to one embodiment may be used to increase the rateat which a user can review the image stream without sacrificing detailsthat may be depicted in the stream. In certain embodiments, the imagesare collected from a swallowable capsule traversing the GI tract.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be understood and appreciated more fully fromthe following detailed description taken in conjunction with thedrawings in which:

FIG. 1 shows a schematic diagram of an in-vivo imaging system accordingto one embodiment of the present invention;

FIG. 2 depicts a portion of a display according to an embodiment of thepresent invention;

FIG. 3 is a flow chart of a method for viewing an image stream accordingto an embodiment of the invention;

FIG. 4A is a schematic depicting a method of generating a single imagefrom four square images according to an embodiment of the invention;

FIG. 4B is a schematic depicting a method of generating a single imagefrom four square images according to another embodiment of theinvention;

FIG. 4C is a schematic depicting a method of generating a single imagefrom four square images according to yet another embodiment of theinvention;

FIG. 5A is a flow chart depicting a method for generating a single imagefrom four square images according to an embodiment of the invention;

FIG. 5B is a flow chart depicting a method for generating a single imagefrom four square images according to another embodiment of theinvention;

FIG. 5C is a flow chart depicting a method for generating a single imagefrom four square images according to yet another embodiment of theinvention;

FIG. 6 is a schematic depicting technique for mirroring edges of images;

FIG. 7A is a schematic depicting a method of generating a single imagefrom four circular images according to an embodiment of the invention;

FIG. 7B is a schematic depicting a method of generating a single imagefrom four circular images according to another embodiment of theinvention;

FIG. 8 is a flow chart depicting a method for generating a square imagefrom an originally circular image;

FIG. 9A is a schematic depicting shape distortion to generate ahexagonal image from an originally circular image, according to anembodiment of the invention;

FIG. 9B is a schematic depicting a method of generating a single imagefrom four hexagonal images according to an embodiment of the invention;

FIG. 10A is a schematic depicting a method of distorting originallycircular images according to an embodiment of the invention; and

FIG. 10B is a schematic depicting a method of distorting originallycircular images according to another embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

In the following description, various aspects of the present inventionwill be described. For purposes of explanation, specific configurationsand details are set forth in order to provide a thorough understandingof the present invention. However, it will also be apparent to oneskilled in the art that the present invention may be practiced withoutthe specific details presented herein. Furthermore, well-known featuresmay be omitted or simplified in order not to obscure the presentinvention.

Reference is made to FIG. 1, which shows a schematic diagram of anin-vivo imaging system according to one embodiment of the presentinvention. In an exemplary embodiment, the system comprises a capsule 40having an imager 46, for capturing images, an illumination source 42,for illuminating the body lumen, and a transmitter 41, for transmittingimage and possibly other information to a receiving device. Typically,the image capture device may correspond to embodiments described in U.S.Pat. No. 5,604,531 to Iddan et al., and to embodiments described inpublished application WO01/65995 to Glukhovsky et al., but in alternateembodiments may be other sorts of image capture devices. The imagescaptured by the imager system may be of any suitable shape including forexample circular, square, rectangular, or hexagonal, etc. Typically,located outside the patient's body in one or more locations are an imagereceiver 12, typically including an antenna or antenna array (notshown), an image receiver storage unit 16, a data processor 14, a dataprocessor storage unit 19, and an image monitor 18, for displaying,inter alia, images recorded by the capsule 40. Typically, data processorstorage unit 19 includes an image database 21.

Typically, data processor 14, data processor storage unit 19 and monitor18 are part of a personal computer or workstation, which includesstandard components such as processor 14, a memory, a disk drive, andinput-output devices, although alternate configurations are possible.Data processor 14 may include any standard data processor, such as amicroprocessor, multiprocessor, accelerator board, or any other serialor parallel high performance data processor. Data processor 14typically, as part of its functionality, acts as a controllercontrolling the display of the images. Image monitor 18 is typically aconventional video display, but may, in addition, be any other devicecapable of providing image or other data. The image monitor 18 presentsthe image data, typically in the form of still and moving pictures, andin addition may present other information. In an exemplary embodiment,the various categories of information are displayed in windows. Multiplemonitors may be used to display image and other data.

In operation, imager 46 captures images and sends data representing theimages to transmitter 41, which transmits images to image receiver 12using, for example, electromagnetic radio waves. Image receiver 12transfers the image data to image receiver storage unit 16. After acertain period of time of data collection, the image data stored instorage unit 16 is sent to the data processor 14 or the data processorstorage unit 19. For example, the image receiver 12 or image receiverstorage unit 16 may be taken off the patient's body and connected to thepersonal computer or workstation which includes the data processor 14and data processor storage unit 19 via a standard data link, e.g., aserial, parallel, USB, or wireless interface of known construction. Theimage data is then transferred from the image receiver storage unit 16to an image database 21 within data processor storage unit 19.Typically, the image stream is stored as a series of images in the imagedatabase 21, which may be implemented in a variety of known manners.Data processor 14 may analyze the data and provide the analyzed data tothe image monitor 18, where a user views the image data. Data processor14 operates software (not shown) that, in conjunction with basicoperating software such as an operating system and device drivers,controls the operation of data processor 14. Typically, the softwarecontrolling data processor 14 includes code written in the C++ language,but may be implemented in a variety of known methods. Data processor 14may include graphics software or hardware.

The image data collected and stored may be stored indefinitely,transferred to other locations, or manipulated or analyzed. A healthprofessional may, for example, use the images to diagnose pathologicalconditions of the GI tract, and, in addition, the system may provideinformation about the location of these pathologies. While, using asystem where the data processor storage unit 19 first collects data andthen transfers data to the data processor 14, the image data is notviewed in real time, other configurations allow for real time viewing.

Typically, the in-vivo imager system collects a series of still imagesas it traverses the GI tract. The images may be later presented as astream of images of the traverse of the GI tract. The in-vivo imagersystem may collect a large volume of data, as the capsule 40 may takeseveral hours to traverse the GI tract, and may record images at a rateof, for example, two images every second, resulting in the recordationof thousands of images. The image recordation rate (or frame capturerate) may be varied.

Typically, the image data recorded and transmitted by the capsule 40 isdigital color image data, although in alternate embodiments other imageformats may be used. In an exemplary embodiment, each frame of imagedata includes 256 rows of 256 pixels each, each pixel including bytesfor color and brightness, according to known methods. For example, ineach pixel, color may be represented by a mosaic of four sub-pixels,each sub-pixel corresponding to primaries such as red, green, or blue(where one primary is represented twice). The brightness of the overallpixel is recorded by a one byte (i.e., 0-255) brightness value.Typically, images are stored sequentially in data processor storage unit19. The stored data is comprised of one or more pixel properties,including color and brightness. Other image formats may be used.

Typically, data processor storage unit 19 stores a series of imagesrecorded by a capsule 40. The images the capsule 40 records, forexample, as it moves through a patient's GI tract may be combinedconsecutively to form a series of images displayable as an image stream.When viewing the image stream, the user is typically presented with oneor more windows on monitor 18; in alternate embodiments multiple windowsneed not be used and only the image stream is displayed. In anembodiment where multiple windows are provided, for example, an imagewindow may provide the image stream, or still portions of that image.Another window may include buttons or other controls that may alter thedisplay of the image; for example, stop, play, pause, capture image,step, fast-forward, rewind, or other controls. Such controls may beactivated by, for example, a pointing device such as a mouse ortrackball. Typically, the image stream may be frozen to view one frame,speeded up, or reversed; sections may be skipped; or any other methodfor viewing an image may be applied to the image stream.

While the following discussion relates to the case where data from acapsule 40 is stored for later use, the system and method of the presentinvention may be used with systems allowing for real time viewing ofimage data.

While, typically, information gathering, storage and processing areperformed by certain units, the system and method of the presentinvention may be practiced with alternate configurations. For example,the components gathering image information need not be contained in acapsule, but may be contained in any other vehicle suitable fortraversing a lumen in a human body, such as an endoscope, stent,catheter, needle etc.

In another embodiment, information gathering can be performed in anothercavity besides a lumen in a human body. An example can includeinformation gathered in an animal lumen. Another example can includeinformation gathered from pipes or other cavities formed during amanufacturing process. Yet another example can be information gatheredthrough a natural stream, for example, geological or marine formations.

Furthermore, while typically the components accepting, processing anddisplaying the image data are contained within a workstation system orPC, other systems may be used, and other (e.g., distributed) componentsmay perform such image accepting, processing and displaying.

In one embodiment, the image stream may be presented to the viewer asmultiple image streams in two or more windows, such that as the imagestreams are displayed a set of consecutive or “nearby” frames aredisplayed substantially simultaneously. For example, in one embodiment,two windows or viewing areas are displayed, each displaying one frame ofan image stream. Typically, the frames are displayed substantiallysimultaneously. According to one embodiment, in each time slot, twoimages which are consecutive in the image stream are displayed, one ineach window or viewing area. For example, in one embodiment, the imagestream is divided into two separate streams that are displayedsubstantially simultaneously. Frames, which are consecutive or adjacentin the original stream, become in the separate streams, correspondingframes (e.g., the first frame in each corresponding stream, the secondframe in each corresponding stream). Thus, when the two resultingstreams are displayed, frames 1 and 2 from the original stream may bedisplayed side by side, then frames 3 and 4, etc. In another embodimentthe image stream is divided into three or more separate streams that aredisplayed substantially simultaneously. In yet another embodiment someframes in the stream maybe skipped.

In an exemplary embodiment, the windows or viewing areas are closetogether, with a minimum of blank or black space between the images, andtypically horizontally and side by side, to allow a viewer to see theentirety of the images without substantially moving his eyes. The imagesmay be distorted (e.g., displayed in a cone, oval or ellipse shapedfield) to further reduce the space between them. The images may bedisplayed with symmetry. For example, the images may be displayed in thesame horizontal plane. One image may be reversed and presented as amirror image, the images may have their orientation otherwise altered,or the images may be otherwise processed to increase symmetry.

Typically, if normally the image stream is displayed at a certain rate,the two separate image streams displayed according to one embodiment mayeach be displayed at half that speed. Thus, if the image stream may bedisplayed at 20 frames per second each of the two streams may bedisplayed at 10 frames per second. In such a case the same number ofoverall frames per second is displayed, but the user can view twice asmuch information for twice as long. The total display time for the imagestreams is the same as that of the original image stream, but each frameis displayed to the user for a longer period of time. In anotherexample, if a user is comfortably viewing a single image stream at onerate, adding a second stream will allow the user to increase the totalreview rate without reducing the time that each frame is displayed. Inalternate embodiments, the relationship between the display rate whenthe image stream is displayed as one image stream and when it isdisplayed as multiple streams may differ; for example, the resultingmultiple image streams may be displayed at the same rate as the originalimage stream.

In an exemplary embodiment, the user may switch modes, between viewingthe images as one stream and viewing the images as multiple streamsusing a control such as a keystroke or on-screen button. The user maycontrol the multiple streams in a manner similar to the control of asingle stream, for example by using on screen controls. In an alternateembodiment, only one mode may be offered to the user. FIG. 2 depicts aportion of a display according to an embodiment of the presentinvention. Referring to FIG. 2, the display 300 is in multiple imagestream mode. The display 300 may be displayed on, for example, imagemonitor 18. Typically, the display 300 includes a set of image windows302 and 302′, for displaying image streams, and a set of controls 304.The controls 304 may include, for example, a toggle control 306,allowing the user to toggle between a multiple image stream mode and asingle image stream mode. The controls 304 may also include, forexample, conventional video controls, such as pause, stop, play,fast-forward, reverse, etc. In a typical embodiment, if the system is ina multiple image stream mode, the controls 304 act on all image streamssimultaneously; in alternate embodiments, other methods may be used, forexample the controls 304 may act on a single image stream.

As seen in FIG. 2, two image streams are displayed. Typically, at anyone time, since the images in each image stream are substantiallyadjacent, since adjacent images are typically similar, and since eachimage stream is displayed synchronously (e.g., the frame having the sameor similar frame number is displayed for each), the images displayed inwindow 302 and 302′ are substantially similar. Typically, a user viewingmultiple image streams directs the center of his vision to a pointin-between the image streams, e.g., point 308. Typically point 308 isnot displayed on the monitor 18 (but may be); point 308 is included inFIG. 2 for illustrative purposes. The user may absorb the relevantinformation about the image streams in such a manner; such viewing mayrequire a period of training. For example, if the image streams are ofthe GI tract, the user may, by directing his gaze to point 308, absorbinformation regarding pathologies from the set of image windows.

In alternate embodiments, the different image streams may be placed indifferent configurations on a viewing screen. For example, rather thanhorizontally, the image streams may be arranged vertically ordiagonally. In further embodiments, different numbers of image streamsmay be displayed. For example, if three image streams are to bedisplayed simultaneously, frames 1, 2 and 3 of the original stream maybe displayed, then frames 4, 5 and 6, etc. In further embodiments,adjacent frames need not be displayed, and the frames may not bedisplaying the specific patterns discussed herein. For example, certainframes may be skipped: frames 1 and 6 may be displayed, then frames 3and 8, etc. Frames that are substantially adjacent, rather thanimmediately adjacent (e.g., frames 1 and 5), may be displayedsimultaneously. The frames in different image streams need not bedisplayed simultaneously; the frames may be displayed at different timesor independently.

Various methods may be used to separate an original image stream intoone or more secondary image streams to be displayed. In one embodiment,images from the original image are simply directed to the proper screenposition at viewing time, and the image stream is not actuallyseparated; in alternate embodiments, the images may be separated forexample, placed in different files or memory blocks. In one embodiment,each resulting image stream includes a separate subset of images fromthe original image stream; in alternate embodiments, the images fromeach resulting image stream may overlap. Subset image streams mayinclude different images from the original stream in differentembodiments.

In certain embodiments of the present invention, more than one imagestream may be collected. For example, an in-vivo vehicle may includemore than one imager (or one imager) collecting multiple imagestreams—possibly by including an imager or lens system in more than onelocation on the vehicle. Capsule 40 may include more than one imager 46.The imagers 46 may be arranged, for example, at either end of thecapsule 40, or at the same end of the capsule, in slightly differentpositions or different angles. A capsule which includes a plurality ofimagers is described, for example, in International Publication NumberWO 02/054932 which is assigned to the common assignee of the presentapplication. Each imager 46 may capture images and transmit the imagesvia the transmitter 41 or via separate transmitters. Typically, eachimager 46 has associated an optical system. In such a case, anembodiment of the system and method of the present invention may displayeach image stream simultaneously, where each image displayed on theviewer screen was typically captured at the same time. In oneembodiment, images from each of the imagers can be displayedsubstantially simultaneously so that image streams from differentimagers can be reviewed simultaneously. In another embodiment, imagestreams from each imager may be divided into a number of subset imagestreams and the subset image streams for one or more imagers may beshown substantially simultaneously. E.g., one subset image stream mayinclude every other image frame whereas the second subset stream mayinclude every other consecutive image frame (e.g. the first subsetincludes frames 1,3,5 etc and the second subset includes frames 2,4,6etc.). Typically, in such a case images may be shown in matrix form sothat each column may display frames from a single imager and each rowmay display frames from a different imager. Alternatively, each row maydisplay frames from a single imager and each column may display framesfrom a different imager. In further embodiments, an image stream may bedivided up or partitioned into sections, rather than based onsubstantially adjacent frames. For example, an image stream may bedivided into a first section and a second section, where the first andsecond section are sequential. The two sections may be displayedsimultaneously. More sections or partitions may be created.

According to an embodiment of the invention a method for viewing animage stream includes the step of displaying an original image stream asa plurality of image streams. Reference is now made to FIG. 3, whichschematically illustrates a method according to one embodiment of theinvention. According to one embodiment an original stream of images isobtained (step 310), for example, by using images obtained by an imagingcapsule that traverses the GI tract while acquiring images of the GItract. The original image stream is then divided into a plurality ofsubset image streams (step 320), each subset including at least aportion of the original image stream. According to one embodiment,frames, which are consecutive or adjacent in the original stream, becomein the separate streams, corresponding frames (e.g., the first frame ineach corresponding stream, the second frame in each correspondingstream). Thus, when the two resulting streams are displayed, frames 1and 2 from the original stream may be displayed side by side, thenframes 3 and 4, etc. The subset streams are displayed on a display (step330) preferable substantially simultaneously, typically for observingand/or analyzing, for example, for detecting pathologies in the GItract. In one embodiment of the invention, the main display engine isprogrammed to select the next two images to be displayed substantiallyat the same time (instead of the next image), and a display windowdisplays those two images at the substantially the same time, until thenext two images are selected from the display engine.

In an exemplary embodiment, the windows or viewing areas are closetogether, with a minimum of blank or black space between the images, andare typically horizontally and side by side, to allow a viewer to seethe entirety of the images without substantially moving his eyes. Theimages may be distorted (e.g., displayed in a cone, oval or ellipseshaped field) to further reduce the space between them. The images maybe displayed with symmetry. For example, the images may be displayed inthe same horizontal plane. One image may be reversed and presented as amirror image, the images may have their orientation otherwise altered,or the images may be otherwise processed to increase symmetry.Typically, a tool available to the user which manipulates an image(e.g., region of interest or zoom) will have an identical effect on allimages simultaneously displayed. Each image may be displayed withdifferent post-processing. For example, one image may be subject tocertain filtering or manipulation (e.g., red or green filtering,contrast enhancement, brightness alteration) and the other image may besubject to different or no filtering or manipulation.

In one embodiment two or more images in a stream or in a plurality ofstreams displayed substantially simultaneously are fused together anddisplayed as a single entity. As such, a user may comfortably andconcurrently incorporate information shown in each of the images whileavoiding the distraction caused by the typically sharp contrast betweenconnecting edges or between the images and the background color whichmay appear between the images when the images are spaced apart.

According to some embodiments fusing of independent images may beaccomplished by, for example, one or more post processing algorithmsknown in the art, including but not limited to, smoothing convolution,mirroring, overlapping, linear or non-linear fade-out fade-in,truncation, linear shape distortion, non-linear shape distortion,normalization or intensity, or other suitable post-processing.

In one embodiment, a relative position for two or more images may bedefined, and juxtaposed edges of images may be defined. A smoothingconvolution on the juxtaposed edges may be performed, and the images maybe displayed with the smoothing convolution. A fade-out in intensity maybe performed substantially near the juxtaposed edges, and the images maybe at least partially overlapped to obtain an overlapping region. In oneembodiment, the overlapping region may be defined as a sum of juxtaposededges, and the intensity in the overlapping region may be normalized.The juxtaposed edges may be mirrored, and a fade-out in intensity on themirroring regions may be performed, obtaining faded-out mirroringregions.

Reference is now made to FIG. 4A, a schematic showing how a new image420 may be generated from original images 412, 414, 416, and 418displayed substantially simultaneously, according to one embodiment ofthe invention. Images 412, 414, 416, and 418 are fused to each other toform a larger square. In one embodiment, post processing is performedover a predefined area 415 around the touching edges, for exampletouching edge 411 between original images 412 and 414. In otherembodiments post processing may be performed on all the original imagesin their entirety. In yet another embodiment, no post-processing isperformed. FIG. 5A describes an algorithm that may be used in oneembodiment of the invention. In step 510, the images to be displayedsubstantially simultaneously are selected. This step may includeselecting the number of images to be displayed substantiallysimultaneously from a stream of images when images are displayedchronologically, selecting the number of images from more than onestream of images, and/or selecting images from a stream or a subsetstream of images based on other criteria for example based on spectralcharacteristics of images (e.g. intensity, saturation, hue, etc). Instep 515, the position of the images used to form a fused image in adefined geometrical shape (eg. square, rectangle, cross, etc.) isdefined. Based on the positioning of each image, the at least partiallyconnecting edges between two substantially juxtaposed images are definedin step 520. Smoothing convolution is performed on defined connectingedges in step 525 and the resultant fused image is displayed in step530. In another embodiment of the invention, smoothing convolution isnot performed. Other algorithms besides and in addition to smoothingconvolution may be used as well including mirroring, overlapping, linearor non-linear fade-out fade-in, truncation, etc. In another embodimentof the invention, post-processing algorithms need not be used.

FIG. 4B is a schematic showing how a new image 430 may be generated fromoriginal images 412, 414, 416, and 418 displayed substantiallysimultaneously in one embodiment of the invention. Images 412, 414, 416,and 418 are partially overlapping to form a larger square. In oneembodiment, post processing is performed in overlapping regions 413′,415′, 417′, 419′, 440′. FIG. 5B shows an alternate algorithm whereimages are positioned to overlap, according to an embodiment of theinvention. In this exemplary algorithm, images are selected in step 535,for example, as described above. A fade-out in intensity is performed onedges in step 540. Images are positioned so that fade-out regionsoverlap in step 545. The overlapped regions are defined as the sum ofthe two overlapped images (step 550). Normalization on the intensity isperformed, for example, in region 440′, where more than two overlappingregions are summed to define an area (step 555). According to otherembodiments other algorithms may be used.

FIG. 4C, is a schematic showing how a new image 425 may be generatedfrom four images 412, 414, 416, and 418 positioned substantially closeto each other and separated by space 422, according to an embodiment ofthe invention. In one embodiment, original images 412, 414, 416, and 418may stay intact so that no information is lost and area 422 is filledbased on post processing performed on image content proximal to area422. In another embodiment, when the juxtaposed images are positionedwith a space between them (e.g., a pre-defined space, or another space),a background color may be used to reduce the contrast between theparallel running images. The background color may be a solid color or astream of colors. The background color may be a stream of colors thatgradually changes from the colors appearing on one edge of a first imageto the colors appearing on the juxtaposed edge of a second image. Inanother embodiment the edges of the images are also processed such assmoothed. Other methods may be used such as mirroring of images. FIG. 6shows an image 615 with edges 617 and 618 where mirrored edges 617′ and618′ were generated. FIG. 5C describes an exemplary algorithmincorporating mirroring that may be used in one embodiment of theinvention. Images to be displayed substantially simultaneously areselected in step 560, for example as described above. Position of eachimage is defined such that the distance between substantially juxtaposedimages equals the width of the mirrored edges (step 565). The juxtaposededges are defined (step 570) and are mirrored (step 575). Fade-out inintensity is performed on mirrored edges (step 580). The area betweenoriginal images is defined as the sum of overlapping mirrored edges(step 585). A space between two or more images may be filled with one ormore colors.

FIGS. 4A, 4B, and 4C show exemplary embodiments using four originalimages to form one new image, however, more or less than four originalimages displayed substantially simultaneously may be used to generate anew image. The new image in one embodiment of the invention may begenerated from more than four images. Original images may be of a shapeother than square for example circular, rectangular, or hexagonal. FIG.7A shows a schematic of four circular images 752, 754, 756, and 758.Images 752, 754, 756, and 758 are positioned such that defined hexagonalareas 752′, 754′, 756′, and 758′ inside corresponding circular imageshave touching edges, for example touching edge 759 between hexagonalarea 754′ and 758′. As such overlapping areas 753, 755, 757, and 759 arecreated. In this embodiment of the invention post processing may beperformed over overlapping areas 753, 755, 757, and 759 to create a newimage incorporating original images 752, 754, 756, and 758.

FIG. 7B shows an alternate embodiment where circular images 752, 754,756, and 758 are positioned such that defined hexagonal areas 752′,754′, 756′, and 758′ inside corresponding circular images would haveover lapping area 750 around the touching edges. In this embodiment,post processing may be performed, for example on overlapping area 750.

In an alternate embodiment when working with circular images, loss ofinformation due to truncation or overlapping may be reduced or alltogether prevented by distorting the original circular image into analternate shape such as for example a square, hexagon, octagon, sector,or any other symmetrical or non-symmetrical shape. FIG. 8 describes anexemplary algorithm of how a square image may be generated from anoriginally circular image using non-linear shape distortion. In block810 a circular image is defined by a function F(x,y) in Cartesiancoordinates. The function is transformed into polar coordinates F(r,theta) in block 820. A ring along the outer perimeter of the circle isdefined in block 830. The image data contained in the ring is remappedto in parts of an unfilled square in order generate a square perimeterin block 850. As such the central part of the image stays in tact andonly a defined perimeter is distorted to generate a desired shape. Othershapes may be generated by partially distorting an originally circularimage using this method.

In another embodiment of the invention linear distortion is accomplishedby defining an array of image data along a radius and re-mapping thearray of image data along the direction of the defined radius. Thismethod of distortion may be performed at every angle or at some range ofangles of the originally circular image in order to generate an imagewith an alternate shape. In some embodiments an originally circularimage may be almost fully distorted. Other methods may be used as well.In one embodiment of the invention, an original circular image isdistorted into a hexagonal shape as is shown, for example, in FIG. 9A.Original circular image 960 may be stretched and distorted to generate alarger hexagonal image 962. FIG. 9B shows an example of four juxtaposedhexagonal images 963, 965, 967, and 969 distorted from originallycircular images. Images 963, 965, 967, and 969 may be used to create asingle image by, for example, smoothing in the area near the touchingedges for example touching edge 966. In another embodiment of theinvention, a hexagon may be truncated from an originally square,circular, or rectangular image. In yet another embodiment of theinvention, a new image is generated from substantially more than fourhexagonal images positioned in a honeycomb structure. Distortion may beperformed along the entire perimeter of the originally circular image960, such as in distorted image 962 or in an alternate embodiment,partial distortion may be performed (not shown) for example distortionto generate one or more sides of a hexagon.

In another embodiment of the invention, images may be distorted toshapes other than square or hexagonal. For example, an originallycircular image may be distorted to an octagon, a sector or near sectorshape or to any other symmetrical or non-symmetrical shape. Distortionmay or may not include truncation, smoothing convolution, re-mapping orany other method known in the art. FIG. 10A shows four exemplaryjuxtaposed images 1010, 1020, 1030, and 1040 that were distorted fromoriginally circular images 1010′, 1020′, 1030′, and 1040′. In thisexemplary embodiment, the image distortion moves the image data towarddead space 1050 that is also the center of view. The originally circularimages 1010′, 1020′, 1030′, and 1040′ are zoomed in the periphery aswell so as to create emphasis in the area away from the center of view.Smoothing convolution, mirroring, overlapping, linear or non-linearfade-out fade-in, truncation, or any of the methods described above maybe used to fuse the distorted images 1010, 1020, 1030, and 1040 onceformed. The resultant fused image in this exemplary embodiment is aflower shaped image, but may have other shapes (e.g., honeycomb, etc.).

In another exemplary embodiment shown in FIG. 10B four juxtaposed images1015, 1025, 1035, and 1045 are arranged in an oblong fashion. In thisexemplary embodiment, the distorted images 1025 and 1045 generated fromthe originally circular images 1020′ and 1045′ are of a different shapethan the distorted images 1015 and 1035 generated from the originallycircular images 1015′ and 1035′. In this exemplary embodiment the imagedistortion serves to move the image data toward the dead space 1055 thatis also the center of view. The originally circular images 1015′, 1025′,1035′, and 1045′ are zoomed in the periphery as well so as to createemphasis in the area away from the center of view. Smoothingconvolution, mirroring, overlapping, linear or non-linear fade-outfade-in, truncation, or any of the methods described above may be usedto fuse the distorted images 1015, 1025, 1035, and 1045. The resultantfused image in this exemplary embodiment is a flower shaped image.

Images displayed substantially simultaneously may be a subset of imagestaken from a single stream of images or from more than one stream ofimages. Images displayed substantially simultaneously may be sequentialimages in a stream based on chronological order or may be arranged inanother order. In one embodiment images displayed substantiallysimultaneously may be selected based on spectral characteristics of eachimage displayed, for example intensity, hue, etc. More than onecharacteristic may be used to determine the order or positioning of theimages displayed substantially simultaneously. For example, within acertain time frame, images from a stream may be selected to belong to aspecific subgroup displayed substantially simultaneously based onminimal color variance. The larger image generated from two or moreimages may be streamed like a video where images are replaced at somegiven frequency or may be scrolled to move horizontally or verticallyacross the monitor.

In one embodiment the user can select the number of images incorporatedin the larger image, the frequency at which the image is updated, andthe type of viewing (e.g. video, horizontal scrolling, verticalscrolling, etc.).

In another embodiment a still image can be displayed substantiallyjuxtaposed to the image streams so that the image streams can be easilycompared to the still image during the viewing process. In yet anotherembodiment, an image stream obtained form an imager is displayedsubstantially simultaneously and substantially juxtaposed to an imagestream obtained from an ultrasound, temperature and/or a pH imager.Displaying image streams or single images substantially simultaneously,from different types of imagers may help to draw a users attention toframes that may show abnormalities and/or pathologies.

In alternate embodiments, the system and method of the present inventionmay operate on image streams having alternate formats, on image streamsbeing from alternate sources than a capsule, and containing images fromsites other than the GI tract.

Embodiments of the present invention may allow an image stream to beviewed as a smoother moving image, with less jitter by reducing the rateat which each image in the set of images displayed substantiallysimultaneously are streamed, without decreasing the original streamrate. In some cases, if concurrently displayed images differ in acertain manner, the viewer may be able to more easily spot pathologiesor other conditions.

In one embodiment of the system and method of the present invention, animage stream is displayed to the user in a manner reducing jitter,smoothing the presentation, and allowing easier and more pleasantviewing of the image stream. In such an embodiment, additionalintermediate images are formed and inserted into the image stream, theintermediate images being formed from combinations of adjacent orsubstantially adjacent images. In one embodiment, the combinations arecreated by morphing or averaging adjacent images so that the resultingadditional image is, when viewed as part of a moving image stream,somewhere intermediate in time to the two original images. In alternateembodiments other methods of combining images may be used. In oneembodiment, synthesizing methods such as those used in theMotionPerfect® software provided by DynaPel of Germany may be used. Suchmethods use motion information from the moving image stream tointerpolate data for additional image frames. Other interpolation orsynthesizing methods may be used.

The system and method of the present invention may allow an image streamto be viewed in an efficient manner and over a shorter time period.

It will be appreciated by persons skilled in the art that the presentinvention is not limited by what has been particularly shown anddescribed hereinabove. Rather the scope of the invention is defined bythe claims that follow:

1. A method for displaying an image stream obtained from a swallowablecapsule as a stream of fused images, the method comprising: creating anoriginal image stream from images received from the swallowable capsule;and displaying images from the original image stream across a pluralityof consecutive time slots; wherein in each time slot a set ofconsecutive images from the original image stream is displayed, eachimage within the set being displayed at the same time; wherein the setof consecutive images displayed in each time slot is immediatelyconsecutive, in the original image stream, to the set of consecutiveimages in the immediately previous time slot; and wherein the set ofimages displayed in each time slot is fused together; thereby increasingthe rate at which the original image stream can be reviewed withoutreducing image display time.
 2. The method according to claim 1comprising generating at least one substantially fused image from atleast two images.
 3. The method according to claim 2 wherein the atleast two images appear consecutively in the original image stream. 4.The method according to claim 2 wherein the at least two images share atleast one spectral characteristic.
 5. The method according to claim 1wherein the images in the original image stream have a shape selectedfrom the group consisting of: square, rectangle, hexagon, octagon, andcircle.
 6. The method according to claim 1 comprising performing on atleast one section of one image a process selected from the groupconsisting of: smoothing convolution, mirroring, overlapping, linearfade-out fade in, non-linear fade-out fade-in, truncation, linear shapedistortion, non-linear shape distortion, and normalization on intensity.7. The method according to claim 1 wherein consecutive images from theoriginal image stream are displayed side by side.
 8. The methodaccording to claim 1 comprising displaying at least two images from theoriginal image stream substantially simultaneously and at leastpartially overlapping.
 9. The method according to claim 1 wherein eachimage displayed simultaneously is displayed with a pre defined spacebetween the images.
 10. The method according to claim 1 comprising:performing fade-out in intensity substantially near juxtaposed edges ofimages from the original image stream which are displayed substantially;at least partially overlapping the images displayed simultaneously,thereby obtaining at least one overlapping region; and normalizingintensity in said overlapping region.
 11. The method according to claim1 comprising: mirroring juxtaposed edges of images displayedsimultaneously from the original image stream.
 12. A system fordisplaying an image stream as a stream of fused images, the systemcomprising: a swallowable capsule; a display; and a processor to createan original image stream from images received from the swallowablecapsule, to display on the display images from the original image streamacross a plurality of consecutive time slots, wherein in each time slota set of consecutive images from the original image stream is displayed,each image within the set being displayed at the same time, wherein theset of consecutive images displayed in each time slot is immediatelyconsecutive, in the original image stream, to the set of consecutiveimages in the immediately previous time slot, and wherein the set ofimages displayed in each time slot is fused together, thereby increasingthe rate at which the original image stream can be reviewed withoutreducing image display time.
 13. The system of claim 12 wherein theprocessor is to generate at least one substantially fused image from atleast two images.
 14. The system of claim 13 wherein the at least twoimages appear consecutively in the original image stream.
 15. The systemof claim 13 wherein the at least two images share at least one spectralcharacteristic.
 16. The system of claim 12 wherein the images in theoriginal image stream have a shape selected from the group consistingof: square, rectangle, hexagon, octagon, and circle.
 17. The system ofclaim 12 wherein the processor is to perform on at least one section ofone image a process selected from the group consisting of: smoothingconvolution, mirroring, overlapping, linear fade-out fade in, non-linearfade-out fade-in, truncation, linear shape distortion, non-linear shapedistortion, and normalization on intensity.
 18. The system of claim 12wherein consecutive images from the original image stream are displayedside by side.
 19. The system of claim 12 wherein the processor is todisplay at least two images substantially simultaneously and at leastpartially overlapping.
 20. The system of claim 12 wherein the processoris to mirror juxtaposed edges of the consecutive images from theoriginal image stream which are displayed substantially.